Catastrophic interference in connectionist networks
نویسنده
چکیده
Introduction Catastrophic forgetting vs. normal forgetting Measures of catastrophic interference Solutions to the problem Rehearsal and pseudorehearsal Other techniques for alleviating catastrophic forgetting in neural networks Summary
منابع مشابه
Catastrophic Interference in Connectionist Networks: Can It Be Predicted, Can It Be Prevented?
Catastrophic forgetting occurs when connectionist networks learn new information, and by so doing, forget all previously learned information. This workshop focused primarily on the causes of catastrophic interference, the techniques that have been developed to reduce it, the effect of these techniques on the networks' ability to generalize, and the degree to which prediction of catastrophic for...
متن کاملDynamically constraining connectionist networks to produce distributed, orthogonal representations to reduce catastrophic interference
It is well known that when a connectionist network is trained on one set of patterns and then attempts to add new patterns to its repertoire, catastrophic interference may result. The use of sparse, orthogonal hidden-layer representations has been shown to reduce catastrophic interference. The author demonstrates that the use of sparse representations may, in certain cases, actually result in w...
متن کاملUsing Semi-Distributed Representations to Overcome Catastrophic Forgetting in Connectionist Networks
In connectionist networks, newly-learned information destroys previously-learned information unless the network is continually retrained on the old information. This behavior, known as catastrophic forgetting, is unacceptable both for practical purposes and as a model of mind. This paper advances the claim that catastrophic forgetting is a direct consequence of the overlap of the system’s distr...
متن کاملSynfire Chains and Catastrophic Interference
The brain must be capable of achieving extraordinarily precise sub-millisecond timing with imprecise neural hardware. We discuss how this might be possible using synfire chains (Abeles, 1991) and present a synfire chain learning algorithm for a sparsely-distributed network of spiking neurons (Sougné, 1999). Surprisingly, we show that this learning is not subject to catastrophic interference, a ...
متن کاملCatastrophic Interference is Eliminated in Pretrained Networks
When modeling strictly sequential experimental memory tasks, such as serial list learning, connectionist networks appear to experience excessive retroactive interference, known as catastrophic interference (McCloskey & Cohen,1989; Ratcliff, 1990). The main cause of this interference is overlap among representations at the hidden unit layer (French, 1991; Hetherington,1991; Murre, 1992). This ca...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2004